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ABSTRACT 

The large-scale development of Time^Shared^ 
Interactive^ Computer-Controlled, Information Television (TICCIT) at 
Brigham Young University is described in this paper. The TICCIT 
project was designed to provide a market success example for 
computer-assisted instruction, particularly for junior or community 
colleges. The project incorporated a combination of computer 
terminals and television consoles that were able to provide the 
learner with a complete instructional system. This paper discusses 
the relationships between the general elements of the system: design, 
training, courseware, and management. The system is unique because it 
incorporates industrial techniques to a large instructional system. 
(MC) 
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I NTRODUCT I ON 

The avowed purpose of the T ICC IT* project Is to provide a market success 
for CAI. The target population Is junior or community college. Courses are 
being developed in freshman English composition, remedial English, algebra, 
and pre-calculus mathematics. Hopefully,, the CAI materi al s -and the system 
will be vastly improved over other CAI capabilities at a reasonable cost to 
educational institutions. 

The Impact at Brigham Young University Is one of a large-scale develop-, 
ment project profoundly affecting the theory, management, and applications of 
basic factors of both instructional development and related research. Structuring' 
instructional material for use In a computer medium has generated some serious 
considerations of the "building blocks" of Instruction. Since computers are 
not intelligent, and are quite arbitrary, new ways' have had to be devised to 
show just what subject matter consists of and how it might properly be manip- 
ulated. These efforts have resulted in theory universally applicable to 
Instructional design. 

Because the project is large-scale, methods have had to be devised 
for generating products according to schedule. The synthesis of management 
procedures thus derived have shown great promise for other development projects, 
large or small. For this reason, the TICCIT project should be of Interest to 
Instructional psychologists everywhere.. 

This paper attempts to give the flavor of the TICCIT Courseware project 
(as opposed to hardware and software), not so much in terms of the mechanics 
and curriculum, but as an example of a development project from which many 
things may be learned. The project is big enough and long enough (until 1976) 
that the experience is bound to cause waves throughout the discipline. 
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IMPACT OF TIME ON DEVELOPMENT 
Time Is an Important enemy In instructional development. Even if 
all involved— instructional psychologists, subject matter experts, or pro- 
duction people, are all fully qualified with a complete sense of purpose' 
and direction — time seems to be a critical factor. Abedor (1972) reported 
a serious reluctance by developers to adhere to disciplined formative eval- 
uation procedures, partly because of the time involved. Diamond, p. 75 
(1971) in the report of a large development project for a music course showed 
time to be critical In several dimensions. This development project encom- 
passed over three semesters of time, with 1-8 listed participants using a 
systems model.. Experience has shown that even small projects, Involving 
an Instructional developer as project coordinator, perhaps one or two subject 
matter experts, and only three or four hours, of Instruction, will take two 
semesters. 

The problem is compounded In an unusual, but not unexpected way. 
Instructional development is a relatively new discipline (Lee, 1972),. which 
means that most involved are in an accelerated learning process. There 
appear to be as many models as there are project coordl nc'tors , In most 
instances, then, the project coordinators are learning in part by doing. 
Graduate students, at least, may be required to develop one instruct iona I I y 
designed and validated product as part of their masters or doctoral programs, 
and then delve into research. The overall effect Is that given organizational 
and Individual change, personnel turnover, and the long time It takes to 
put a product together, few qualified people stay In one place long enough 
to go through a second deve lopment ' project ^ let alone a series of them. ' 

Another major factor Is the seem'ingiy i nseparab le re 1 at lonsh Ip between 
instructional development and research. As one moves through a -deve I opment 
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project, even though well modeled and scheduled, questions continually arise 
for which there are no ready answers. For example, would it be advantageous 
given the target"population to include audio? Or which test mode would be 
best? Or what sequence should be used? These illustrations, of course, 
are oversimplifications of the real problems. However, each time such a 
question comes up, there is an agony of time involved in either reviewing 
the 1 iterature, generating a research project, or forging ahead on a "gut" 
feeling with the expectation that mistakes in judgment wil I be corrected 
during formative evaluation. Obviously, all of these take time. 

INTRODUCTION TO THE TICCIT^ COURSEWARE PROJECT 
Three factors at Brigham Young University have changed drastically • 
some preyj_o,usly held concepts about instructional development, and research 
as well. These are: (I) a synthesis of ideas about a basic theory of 
instruction which allows developers to approach content and learning strat- 
egies as individual entities (Merrill and Boutwell, 1972), (2) development 
of basic notions about learner-control oriented towards allowing students 
to decide in large measure both content and mode of presentation (i.e., 
whether the student will be given statements of fact, practice, definitions, 
rules, or other available options at the easy or hard level, and in what 
sequence (Bloke, 1972; Bunderson, 1972; and Fine, 1972)), and (3) the impact 
of the TtCCIT^ project, which, if it did not catalyze the other two factors, 
at least benefited immeasurably from them in the same time period. 

Team approaches, or systc approaches, are wt new in instructional 
development or research. The project by Diamond (1971) and his associates 
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is a good example of the use of many people in a significant development 
project. Lee (1972) In his field study cites many more. Both stress the 
importance of teamworl< and a systematic approach rather than just getting 
a group together. The TICCIT Courseware project at BYU, however, is a 
very large project from which many lessons in teamwork, appi i cat ions of 
theory, and production accountability have been learned. This paper attempts 
to describe some of them, not as a matter of research, but rather as a result 
of experience. The observations, of necess i ty, _ are qui te subjective. 

The genesis of the TICCIT Courseware project is described to some 
degree by Bunderson (1972). There has been too little published on the TICCIT 
project, so that Information about it has spread quite largely, by word of 
mouth. The first national article was written by Thomas D. Proffitt (1972) 
a subject matter expert assigned to the project. Later, an. article appeared 
in Science magazine' (Vol. 176, June 9, 1972) which described both the TICCIT 
project and the Plato IV project at the University of Mlinois. Both of 
these projects are funded by National Science Foundation. A basic difference 
Is that the TICCIT project relies on mi n i -computers, or a decentralized 
approach, while the Plato IV system uses a' large central computer. Course-- 
ware In each instance is quite different. 
Courseware 

An explanation of use of the term "courseware" should be helpful. In 
computer parlance the terms hardware and software are quite well known. 
However, In discussions concerning media, many of the products are called 

'ihere appears to be one glaring error in the article regarding dis- 
placement of teachers. The intent of TICCIT Courseware Is to change the role 
of the teacher to be more interactive, perhaps with the ability to handle more 
students in a better way, rather than to replace them. 

O 
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"software." 'To avoid this confusion, and, as a matter of fact, as a quite 
precise term for educational material apart' from media, the term '^courseware" 
has proved to be very viab le. 

One Important aspect of courseware is that it is basically universal. 
Good courseware is needed in any medium, and good courseware by adaptation 
can be used in any medium. For example, the materia! being generated for 
the TICCIT project could be adapted for' use with the Plato IV system, textbook 
applications, movies, videotapes, tape-slide, lecture, or any other media 
mode. One of the expectations of the TICCIT Courseware project is that, by 
■the use of good courseware, as well as an efficient and economical system, 
many of the earlier deficiencies of CAI with dependence on programmed inst rue-- 
tlon can be overcome. Earlier failures in educational TV, CAI, and other 
innovative techniques, including media applications, may very well have 
been founded in the use of incorrectly des i gned • courseware. 
TICCIT at BYU 

Brigham Young University is a subcontractor to the 'MITRE Corporation 
of McLean, Virginia, who, in turn, has contracted with National Science 
Foundation for the TjCCJT system, at an overall cost exceeding $4 million. 
MITRE is providing the hardware and software, BYU the courseware. The latter 
contract extends into 1975 at a cost of over $1.2 million. The .system basically 
consists of two Data General NOVA 800s, with specialized peripheral equipment, 
driving up to 128 student terminals simultaneously.' These terminal s consist' 
of a special ly modified Sony Trinitron color TV set with a special keyboard. 
There are seven colors used on the displays. The center of the xeyboard is 
an, I BM~Selectr ic type, adaptable for mathemat i cs and other special uses. 
There are fifteen keys to the right, twelve to the left for learner control 
(Blake, 1972). 
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The courseware consists of two complete Instructional systems in 
English composition and mathematics. The material involved Is equivalent 
to approximately six semester hours of instruction in algebra and pre-calculus 
mathematics and a similar number of credit hours In remedial English and. 
freshman composition. The target population is that of Junior Colleges or 
Community Colleges having an open-door admissions policy. It is estimated 
that the courseware would cover about 20% of the average Community College 
curricula. 

A remarkable aspect of the Project Is that the elements of hardware, 
software, and courseware are all being developed simultaneously. The effects 
of this are a healthy reluctance by anyone to ^'freeze^' the design prematurely, 
a need for full understanding of the total system by key personnel In every 
area, and an even more critical than usual requirement for adherence to 
1 ndustry^ I i ke management procedures. 

A one-terminal system was established at BYU In December, 1972. 
This is being expanded in March, 1973 to a five-terminal system. By 
September, 1973 there will be approximately 30 terminals In use for computer 
entry of courseware materials, compi I ing, debugging, and format 1 ve eva.I uat ion . 
In the summer of 1974, the BYU system will be moved to Phoenix Junior College 
and expanded to the full 128 terminals. At the same time, a complete system 
will be established at Northern Virginia Community CoM'ege In Alexandria, 
Virginia. At these sites, courses will be conducted for a year or more, 
after which time final clean--up and analyses of the project will be accom- 
plished. Educational Testing Service has a contract to conduct the summative. 
evaluation of the project. 

I 
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COST ANALYSIS 

Obviously, there are numerous functions which make up a large-scale 
development project. Identification of these functions and their refatlon- 
ship and o.rder of magnitude is vital to an assessment of the amount of time 
Involved, the skills which are needed, the dollar costs per category, and 
the phasing relationships among identifiable tasks. There seems to be no 
great question that the functions identified as being important to the 
TICCIT Courseware project are genera I Izab I e to other. devel opment projects, 
regardless of magnitude. Conversely, key functions for other development 
projects are appropriate for conduct of the large-scale TICCIT project. 
These are listed In Figure I. 
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A compounding difficulty In managing these functions is that any 
one of them may be active at any given tJme. However, there are two major 
lessons learned thus far from TICCIT project experience. These are: (I) 
The almost vital necessity for completing essentially the design elements 
before major production phases begin. This Includes all the design with as 
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much emphasis placed on formative evaluation design as any other element. 
(2) The need for training — from instructional psychologists on down. Each 
function has its unique facets which require a disciplined approach, with 
a I arge measure of hofnogene ity . 

The cost data for the TICCIT Courseware project will be provided to 
Educational Testing Service for the summative evaluation, 

TRAINING - PART OF THE SYNTHESIS OF PRODUCTION 
The classic approach to instructional development has been for a 
project coordinator, either a faculty member as subject matter expert or 
an instructional fechno I og ist (a graduate student or instructional psychol- 
ogist), to work with others, quite often in small numbers, to produce 
instructionally designed material.' This is generally accomplished in linear 
fashion. That is, certain individuals perform several of the functions 
previously listedp such as design, authoring, evaluation, and production. 

One of the early steps has been to identify the objectives to be reached 
by the student in behavioral terms. From the vast literature on writing 
behavioral objectives, it Is apparent that these are not easy for even the 
professional to define, let alone a layman. And yet, at the very least, 
there may be an expectation that subject matter experts will somehow provide 
a viable list of behavioral objectives. If this is the case, there is obviously 
a training problem just in beginning production of courseware materials. 

An alternative Is for one skilled in describing behavioral objectives 
to elicit .from the subject matter expert enough information regarding the 

'or. M. David Merri I I , Director, Instructional Research and Develop- 
ment, Brigham Young University, calls this the "artisan or shoe-shop approach." 
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topics to be covered that he can then state them in useful term's. The 

impact of this would be that a qualified instructional design person could 

deal with any subject matter expert to arrive at the behavioral objectives 

for a course. This would alleviate a serious training problem. 

Unfortunately, experience on the TICCIT Courseware project thus far 

has tended toward a much more serious problem than training in behavioral 

objectives, although problems In writing them have been minimized In a way 

which will become clear. It Is generally accepted that prior to addressing 

behavioral objectives, larger goals must be Identified. It may be helpful to 

cite a process which has proved viable for the TICCIT Courseware project, and 

which shows promise of genera I izab i I ity to any other project.' The training ' 

implications of this approach are quite apparent. 

■ Logical' Sequence of Course Organization and Course ware Parts Generation ■ 
— An Authoring Problem 

To stimulate and discipline the th I nk i ng of those involved' in an 
instructional development project, documents ' re I at Ing to the needs, goals, 
and justification should be produced. The individual who should be most 
qualified to perform this task is a subject matter expert with considerable 
experience In his field. He may be assisted by an Instructional psychologist, 
who would provide insight Into possible general instructional strategies. 

Then, it is possible to state a set of mastery models, which relate 
to the type of life condition to be achieved by the student of a given target 
population,. There may be several mastery models. These might be terminal 
objectives, but tend to be difficult to measure in a formative evaluation sense 
because of the length of time generally needed for their achievement. For 



I am indebted to Dr. C. Victor Bunderson, Director, -Institute for 
Computer Uses in Education, for his synthesis of this model, especially 
Q that pertaining to needs, goals, justification, and mastery. 
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example, a student may learn mathematics to enable him to be a good automobile 
mechanic, but will not adopt that mode of life untii he finishes his school ing. 
Mastery models are the primary responsibility of the subject matter expert. 

The author is also primarily responsible for drawing up a list of 
topics to be covered in the course. Note that from this I ist, terminal 
object Ives. cou I d be written, as well as sample test items. Other items may 
also be defined, such as "so what"' approaches, and statements of definition, 
rules, memory items, or problem-solving tasks (generalities). Experience 
has shown that subject matter may be approached from any of these five 
directions, rather than from merely the behavioral objectives. Moreover, 
there is a hierarchy involved in that'all five items relate to each other at 
a specified level of lesson content, and that there are successive levels, 
e.g. course, unit, lesson, and segment. Regardless of from which direction 
approacfied, +he five items and the hierarchies must be furnished. In addition, 
prerequisites must be identified (a matter of insight by a teacher or faculty 
member into the natufe of the -^arget population, the ski I Is involved, the 
probabilities of error and the reasons for error, and the proper hierarchy). ^ 

With the many levels involved, all the foregoing is a repetitious 
task. In addition, it is much more complicated and difficult than merely 
training personnel in stating correct benavioral objectives according to Mager 

'"So what" is a term to describe material oriented toward helping a 
student understand why he must grasp the concept. It is based largely on the 
rationale emanating from the needs, goals, and justification and flowing 
through the mastery mode I (s ) . 

Note that the content oriented materials, such as the needs, goals, 
and justification document; the mastery models; the list of topics; terminal 
objecfives; practice test items; etc. are very valuable for content va I idat ion, 
i.e., submiss ion to a panel of subject matter experts for a subjective deter- 
mination of tentative suitability. Objective determinations should be made 
during format i ve and summatlve evaluation. 



(1962), In spite of the fn^-+ that it has been found to be less productive to 
• state the object!^ Jargon under these circumstances than to 

describe them in ..dent finds palatable and can uncLi stand. 

One other important ingredient must be added in which most textbooks 
and classroom instruction are poor. These are the files of Instances (examples 
and matched non-examples) which go with the generalities (concepts and rules). 
The Instances files must be plentiful enough and divergent enough to allow 
precise learning to occur, and to allow for sufficient practice using prev- 
iously unencountered Instances (to avoid a mere memory process). 

There are additional factors, as welt, to consider, such as the type 
of grade contract, diversionary materia! with high affect (which may or may 
not be relevant), or a I ternate med I a modes* 

Training Is required for allot these tasks. 
Courseware i's Theory Based 

There is one other element of authoring worthy of consideration in 
context with the train ing problem. Stress has been placed on the Idea of 
authoring and content, less on notions of strategy, which may be more 
properly the domain of the instructional psychologist or techn i ci an . The 
theoretical structure of instructional materials are described by Merrill 
and Boutwell (1972) In a taxonomy of instructional variables which was 
summarized by Bunderson (1972) in the following way: 

^'MerrilMs taxonomy involves three classes of variables": 
I. Presentation form Expos*^+o^V ^^qu'^sltorV 

Generalities 

Instances ! 1 

The system deals primarily with concept learning and rule using, 
so a generality Is a definition of a concept, or a rule. An 
O „ instance is an example or non-example of a concept or a rule In use, 

ERjC 
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Expository means to tell, Inqulsitory to ask. Inquisitory gen- 
eralities (e-g-, "define a concept^') are rarely used. 

2. Jnter-disp I ay re I ationsh ins 

This category involves the matching and pairing of examples and 
non-examples, the difficulty levels of instances, the scope of the 
generalities and instances, and their abstractness ~ concreteness . 

3. Mathemagenic Information^ 

This category involves prompting and cuing and other attention- 
focussing techniques. Specific techniques include attribute 
isolation, search strategies, mnemonic aids, and production strategies." 

It is possible, and it is felt desirable, to approach the synthesis of 
courseware from the two directions of content and instructional strategy, 
with the author primarily responsible for content, but the instructional designer 
primarily responsible for strategy. This presupposes an intimacy by the 
designer with the media to be used. There are disciplined procedures and a 
good rationale for this approach covering a thorough content analysis and an 
instructional analysis which help to structure courseware regardless of media 
(Merrill, 1972), The Interactions between a content specialist and an instruc- 
tional specialist in this model may be minimal. However, the point at which 
student learning is rea-l !y focused, if not where It actually occurs, is in 
the domain of mathemagenic information. This is where content and strategy 
mesh by way of the conte>d-ual relevancy of mnemonic aids (for memory Items), 
atrribute Isolation (for concept recognition or classification), algorithms' 
(principle or rule using), or heuristics (for prob I em so I v i ng ) . Merri I I ( 1971 ) 
has described the psychological conditions used here. The mathemagenic 
information also plays a large part in the affective domain. 



^Information which gives birth (gen-) to learning (amthema). 
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Need for Warmth 

In spite of the logic of the foregoing approach;, there is a concern 
that the esF of pleasure and warmth in learning may be lost If content 
and strata;:^' -^^^ ot approached with an undeflnable sense of empathy "and 
humanity. Authoring of courseware' for the TICCIT system, and any other 
courseware, is a disciplined art. Experience until now has shown that, 
contrary to an earlier belief, just any subject matter expert cannot be 
exposed to the wiles of an instructional design specialist and be manipulated 
into producing good stuff In spite of himself. As is true in many Instances^ 
good courseware authors may be born rather than made, even though tralni^ng 
is required in any event. This phenomenon may account for the fact that 
there are few really good teachers in spite of training, or few really good 
textbook writers J 

Other Team Members 

Much has now been said about the problems of t ra i n I ng • sub ject 
matter experts (authors). There are others on the team, such as Iristruc- 
tional psychologists. Instructional design technicians, emp I r i ca I des i gn 
technicians, and packaging specla I ists who need training not only in their 
own discipline, but in the vagaries of courseware unique to the target 
popul ation , the curri cul um, and the medium being used. Training needs for 
these categories of personne I w I I I become clearer as their positions are 
described. 



'The production of good courseware, however, is antithetical to the 
usual textbook authoring approach, although good courseware could be adapted 
readily to the textbook medium. 
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THE FACTORY 

The instructional development factory as exemplified by the TICCIT 
Coursewnre project .incorporates industry techniques in a way readily gen- 

■ to other development, projects, large or sma I I . Variations in 

the model have already been applied to projects at Brigham Young University 
and wi I I continue to be used. One of the apparent major benefits is that 
larger numbers of people, once having been trained, can accomplish several 
projects while the personnel are available rather than having to rely on 
fewer people over a longer period of time^. Other expectations are that 
individual, skills may be better identified. Economics of manpower loading 
may be realized through use of mass-production techniques. Training may 
be simplified and standardized, with a large reliance on functional on-the- 
job techniques rather than massive classroom training sessions, although 
these are not eliminated. 

Personne I 

The TICCIT Courseware project expects a peak of approximately 32 
full-time and 45 ha If-time- personne I to be assigned at any one time. 
These individuals are in various states of training as the many functions 
are conducted, including design, production, and operations and maintenance. 
Since this is a development project, a major objective besides a market 
success for an improved CAl system is to learn how to do this kind of 
development. An overall organizational chart is shown in Figure 2. 
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In addition to the management assignments, TiCCIT courseware pro- 
duction teams use the following functionaries: 

a- Instructional Psychologist (IP) . Rule of thumb is that' one IP 
Is assigned haif-rtlme for each course. His role is to provide advice, 
assistance, coordination, and review of the course material. He bears the 
burden of training, and for some period is involved in some way in the design 
effort. He furnishes the subjective evaluation described by Abedor (1972) 
as the initial professional level screening of finished material. The 
interaction between training and qua I Ity control appears to be direct. That 
is, the better the tra i n I ng/ and thus the qua I i f rcations , of those individuals 
providing inputs, the less lil<ely will be the difficulty in quality control 
review. At least two other aspects of instructional psychology as a discipline 
seem to be highl ighted by the experience with the TICCIT project. These are: 
(I) Whereas usually we speak of instructional psychology as having general 
branches of development and research, the management requirements seem to 

be equally important to any others. With the accountability needed to meet 

Industry-I il<.e deadlines, it may be that only those Instructional psychologists 
who are willing to accept good management roles, and adopt the necessary 
practices, will succeed in what may be a vast area of endeavor In the dis- 
cipline. (2) There is almost a complete lacl< of homogeneity in the thinking 
of credent ia led instructional psychologists. The reasons for this may be 
apparent. As stated earlier, instructional development is a relatively new 
discipline, still groping In many ways for "best" approaches to the science 
of instructional design. However, to be a productive member of an instructional 
development team, much harmon I zat i on wi I | be needed. It may be necessary 
for instructional psychologists in a teamwork approach to undergo further 
training in areas not heretofore studied during a previous career. There Is" 
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also the problem of the type of contribution to be made by an IP, Management 
is certainly an important factor. There are questions concerning the artistic 
input in creating displays and other content structuring which may be in an 
entirely creative realm rather than in a. scientific discipline. Many of '"he 
final decisions regarding these questions may on!v' :. ilHC-iv.V formative 
evaluation and empirical validation, 

b. Author (Subject Matter Expert-SME) , One or more authors are 
assigned to each authoring team. These individuals are expected to be 
wel I -versed in the course subject and are primari ly responsib le for course 
content. They do influence other elements of the course, however, by 
providing advice and coordination at selected phase points during production. 
SME's are generally at various levels of capability as shown in Figure 3, 
which depicts an ideal organ i zat i on for one team assigned to one course. 
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Average annual personne 1 costs for such a team approximate $45,000 
not counting overhead (fringe benefits and facilities costs). The team nn 
produce appr.oximate I y forty-five ono-^^ -if^ \ ^s in a year. The products 
include structured content not yet adapted to the medium, but in an adaptable 
format- 

The actual pT^ducts for which SMEs are primari ly responsible are 
these (Low, i'972); keeping In mind that interaction with an IP or an instruc- 
tional deslgm+echni clan may be required: Hierarchies, topics, objectives, 
generalities. Instances (matched examples and non -examp I es ) in divergent 
order, "so wha*s" ("Why is this mater la I i mportant?^') , prerequ is i tes, helps 
(mathema'gen,Fc treatment) and easy and hard versions of the generalities. 

c- 1: n s t irojct I on:a I De s i pn Te chrn I c t a n (IDT) . Normally one intern is 
assigned h a t f -irlime to^feach teseniiu These are Ph.D. candidates in educational 
psychology who are wairking as graduate assistants in Instructional psychology. 
Each technician ;[s primarily responsible for converting content materiats 
received f rom tte autfer i ng teanns i nt^o a structure compat i b J e with packaging 
needs. The objective is .to provide t^he ;best instructional design for student 
use considering the rredia and: srf rategy options. Products for which the IDT 
(s primari I y responsible are- assist^ance in providing objectives, prerequisites, 
ihterarchies and dispf:a¥^ conventions; response conventions and answer processing; 
display formats; and; :hel ps (mathemagen ic treatment) of instances. 

d. Empirical Design Technicran (EDT) . As with the IDT, one intern 
From the Ph.D. program in i nstruct Iowa 1 psychology is usually assigned half- 
flime to each team. These individuals are responsible for planning and per- 
forming severs:^ types ^ot -format ive evai uat ion of the courseware, dependmg 
on its stage developm»t. The objsctrve. is to assure that the imaterial 
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is suitable for use with the target population* The products are generally 
written summaries of findings of experimental and validation activities. 

e, Packagi ng Spec i a I ist (PS) . These individuals convert authored 
material into a form suitable -^or use in the medium, in this case a computer 
system. They may do this by filling out data forms., or by entering the 
material directly into the system on-line at a student terminal. Other tasl<s 
i ncl ude ed it ing the material, acting as proctors for students on the system, 
performing graphics entry functions, acting as liaison between packaging and 
authoring efforts and acting as a '^sounding board" for the courseware. 

Production Model and Schedule 

As stated by Lee (!972) there is no s I ngl e product ion model. Any 
one proposed must be flexible enough to accommodate the biases of key Individuals 
who are independent and creative, yet rigid enough to allow g'eneratlon of a 
production schedule. The author (Low, 1972) defined a production model which 
accounted for ail. of the end products to be formulated at that time (Appendix). 
The modes, at which major end products were to have been completed were called 
V^stations," so that lesson materia I s cou I d ' be fo I lo.wed through from beginning 
to end i-n a controlled way by referring to station output. The model did not 
include Internal arrangements for authoring teams, which were expected to 
devise their own- "These have since been fonnulated for each of the four teams, 
and as might be expected, no two are precisely alike. 

The overall model allowed eight weeks for completion of a lesson from 
start to f in ish . The master schedu I e called for beginning a lesson each week, 
so that there was cons i'derab I e built-in vertical manpower loading. Of the 
eight weeks, two were allocated to empirical validation at two separate stations. 
One was at a midpoint (Station 10) and the other toward the end (Station 19). 
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These stations are not in the current model. Also not included are many of 
the outputs of Station I 3, such as videotape scripts (which are being handled 
separately), ml n i -I essons, "So What'^ fun opt ions /grade- cent racts , and com- 
plete instance files. These were estimated to require a week. Much of the 
material in Stations 4 and 6 has been combined into Station I. But as 
Bunderson (1972) points out, the basic cycle has been cut to about five 
weeks. The schedule now broadly looks' like Figure 4 for each lesson. 



Through Station 5 (less graphics and tests, 
and not correlated) 

IP review and recycle. 

Team review, correlation, and retrofit 

Packaging teams (coding and graphics) 



two .weeks 



I -1/2 weeks 



one-half week 



one week 



rOTAL: Five Weeks 



Figure 4 



As more and more material Is produced, other- cycles of effort will 
have to be generated. There is still a composite manpower loading to con- 
sider, since more than one lesson Is being worked on at a time, as well as 
different elements or stages of a lesson, unit, or course. 
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CONCLUSIONS 

There are some significant conclusions which may be drawn from the 
experience of the TICCIT Courseware project at Brigham Young University 
thus far. These are categorized as follows; 

Identification of Functions . Figure I contains a list of functions 
identified, as being Important to the analysis of a development project. These 
same functions need to be Identified, with applicable variations, at the 
outset of any project. One major use is that of budgeting and time-phasing. 
Without this information, too little may be known about a project to define 
accurately its management parameters, or the tasks to be performed. 

Des_ign_. This is a critical item. The design elements should be 
completed as early in a projectjas possible. Indeed, the other functions, 
with the possible exception of management and initiation, should not even be 
started until the design Is well on Its way. As In statistics, time spent in 
establishing viable designs will be amortized later on, and help to avoid 
agonies during the production phases. 

Train ing . Another critical item. The disciplined thinking Inherent 
In structuring the training necessary in a project will have wide-spread effects. 
The training establishes a sound basis for quality control. It is also a 
reflection of the design. If at all possible, training packages should be 
prepared as soon as possible as progress In design will allow. There very 
well may be areas of training which are skill oriented rather than project 
taks oriented which could be covered even before or during design. 

Courseware . A theory based approach to content and instruction allows 
for the genera lizabi I Ity of the course materia! across media. This has to be 
considered a breakthrough which could reduce re-inventions of course material 
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everywhere, as well as to tend to standardize curricula. 

Learner Control . !t may be as rmportant for students to learn how to 
learn as It is for them to gain content proficiency. All the familiar terms 
in instructional development, such as "se I f -paced , " "individualized," "non-lock- 
stepped," "teaching toward mastery," "criterion referenced," etc. are embodied 
in the TICCIT Courseware project. Students will have control over both 
content and the way it's presented within medium limitations. The principles 
learned in how to structure material for learner control will have a beneficial 
impact on other instructional development efforts. 

Efficiency . Even though the TICCIT Courseware project is large and 
expensive, much of the effort and money thus far has gone into design and 
procedural matters. Pushing the edge, of theory and technique has cost 
something which should not have to be repeated. Teams can get projects, or 
parts of projects, completed with greater dispatch and less total manhours 
than small groups of i nd i v i dua I s working alone. This is an expectat ion wh i ch 
is expected to be borne out by the summative evaluation. Unfortunately, there 
may be a tendency to compare the costs and efficiency of producing good 
courseware with those of a historical textbook approach, an ever present 
danger. I nst ruct iona I I y designed materials do cost time and money, so that 
high-payoff courses should be first priority. 

Authoring . There is a new brand of authoring required for projects 
like TICCIT. As a discipline, it may be impossible for instructional psy- 
chologists to train enough faculty members, teachers, or even other members, 
to be able to unilaterally redesign or invent good courseware throughout 
education. It will all take tijne and a vast amount of individual training. 
More likely, groups of authors and other skilled persons with' a great deal 
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of aptitude for participation in courseware des-ign will carry the brunt of 

activity in this important area. Just any subject matter expert, or even 

any instructional psychologist, will noi do. • . . . ^ 

Logical Approach to Development Products . A major disciplining factor 
in generating the many development products is the overall approach to content 
and, instruction.- A pattern which starts at a broad, general ^ea, and goes 
by successive logical steps to specific items has been determined to be 
very effective. Work done to describe needs, goals, justification, strategies, 
and resulting mastery models will have a direct application at the basic segment 
level, such as "so what" material, or relevant examples, definitions and practice 
items'. The general series of documents, falling under • Project In it iat ion ' i n 
Figure. I, should be completed very early in the process. To assure widespread 
acceptability of the material, these early documents should be validated for 
content by subject matter expert peer groups before actual development begins. 

Management . Greater insights into the need for good management- in 
Instructional development efforts are required. One of the failings i.n 
educational environments, sometimes under the guise of academic freedom, is 
the lack of accountability on the part of many. One of the reasons for this 
is undoubtedly the creative nature of the individuals i nvo I ved— there is always 
a better way to do whatever is being done. Industry-like practices legislate 
against this phenomenon. Many academicians may have to discip I ine themselves 
as students who have deadi ines for turning in assignments which are going to 
be graded without the potential for amendment. 
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The attached flowchart has been extracted from Low, et. al , August 
I, 1972, revised September 5, 1972. The basic document contains several 
pages of narrative to explain for local use clarifying information about 
each station' and the flowchart in general. A glossary of terms is also 
included. The model is val id as an example of the way In which lesson com- 
ponents can be followed through a multiple production cycle. In this sense, 
it is general izable to'any instructional development project. However, 
experience has shown that models are subject to constant revision and vio- 
lation, and that a viable approach is to formulate a trial model, then 
solicit maximum part icipatior; by those who are going to have to live with 
it. In other words, it should be their model rather than your model . 
Compromises can be made if the schedule is viewed as broad enough to accept 
internal aberrations as long as major deadi ines are met. 
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TICCIT Courseware Project 
Production Model 

The production cyc^le flowphart to follow contains a graphic 
presentation of the material previously given in narrative form. There 
aire several features which should be noted* 

a. The three general phases of activity are shown. 

b. Each station is numbered in parentheses. 

c. The major category of team member is shown within each block, 
e.g., SME. 

d. More detailed team composition is shown to the side of each 
station and correlated by ;3 tat ion number. The team leader in each 
Instance is underlined unless it is a one function team. 

e. The products in succinct terms are specified in each box. 

f. The expected time at each station is shown in the lower right- 
hand corner of each box. 

g. Continuations of the flowchart are shown by a ^ with a letter of 
the alphabet entered for correlation. 

h. A (^^^^ shown by each station at which an IP participates. 
This designates a correction loop which may be specified by the IP to 
any previous station as well as the coordination which may then be 
required. 

it Blocks shown with a dotted line are optional at the discretion 
of the IP. 
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PRODUCTION CYCLE FLOWCHART 



PiMSE 1 



StSE/IDT 



(1) 



Heirarchles , Topics , 
Objectives , Generalities 
Ins tancog # Prac tice 
Exorcisoa , Integrated 
e^^amples and non- 
es^^ainples. 

One Week 



IP 



(2) 



Critique of Station 1 
Output. 



One Dav 



Station Team 



SNEl, SHE2, 
SME3, IDT, 
ED, PS3, SME4 



IP 



PS 



(3) 



Corrected Courseware 
Components 



One Day 




SHH/PS 



Help, Supethelp, Easy 
Hard for Generalltlea. 



Tvo Days 
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IDT/ PS . 



Help, Superhelp and 
Answer Processing for 
Ins tances. 



Two Days 




PS3 



4a SMEl . SME2. 
PS2, SME3, 

SME4 

4b IDT . PS3, 
PS2 
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Station Team 



IP, SME, IDT, ES 



(5) 



I? Review 

Verified Courseware 
Compoaents and T^equire- 
ments. Tests, Art, Audio 
etc. 

Two Days 



IDT . SMEl, SME2, 
PSl, PS2, .PS3, 
PS4, IP 



IDT 

Base Frame 
Specifications 



(6) 



One Day 



IDT 



id 
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(7) 



Courseware 
Samples 



Two Days 




PS2 . PS3, PSl 



IP 


(8) 








Critique 


8 


IP 




One - Day 








PS 




— r Station 



(9) 



Corrected 
Material 



One Day 



PHASE 2 



' EDT 



(10) 



I Reports of Empirical 

I Tryout or Content 

] Validation 
I 



10 



Team 



re2_, PS3, 
PSl 



EDt . PS3 
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I 



One . W«ek 



IP/SME 


(11) 


Directives 


for 


Courseware 


Material 


Completion. 






One Day 




11 



IP, SMEl, 
SME2 



-29- 




N 


t 


PS 


(12) 


Corrected 


Material 

« 




One^ Day 1 



Station 



12 



Team 



PS2 . 
PSl 



PS3, 



PS 


(I3a) 


Instances ; 


Helps and 


Superhelps 


, Practice 


Exercises , 


and Answer 


Processing 


for Instancei 


in volume. 






One Week 

■ 



SME/PS 



(13b) 



Minilesson, "So What" 
Fun Options, Grade 
Contracts, VT Scripts. 



One Week 




13a FS2 , PS3, 
PSA, PSl 

13b SME2 , SME3, 
PS3, PS4, 
SMEl 



ED 



Edited Material 



(lA) 



One Day 
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IP/SME (15) 
Final Critique 



One Day 



Station Team 



15 IP, SMEl 



PS 



(16) 



Final Corrected 
Material 



One Day 



11? 



(17) 



Final 
Coordination 



One Day 



PS 



(18) 



Computer Entered 
Material 



Two Days 




16 PSl . PS2, PS3 



17 IP 



18 PS2 , PS 3 



-31 - 



ERIC 



r 



I EOT 
I 



(19) 



Validation 
Report 



Station 



19 



Team 



EDT, PS3 



L. 



PS 



one week 



r 



(20) 



Courseware 
Distribution 



20 PSl . PS2, 

PS3 
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GLOSSARY 

Algorithm: A rule or procedura for solving a mathematical problem; in a 
broad sense, an iterative rule. 

Answer processing: Analyzing student inputs to determine the correctness 
or incorrectness of the response. 

Assistant author: Graduate level subject matter expert assigned to authoring 

AI: Attribute Isolation. 

Attribute isolation: Process used to illustrate relevant attributes of a 
generality or instance. 

Author: Subject matter expert who writes lesson components and who 
determines content. 

Author assistant: Packaging specialist temporarily assigned authoring duties 
Backward book; Material in sequence from back to front. 

Base frame: Specific data which controls the use of computer subroutine. 

Base frame content file: Wrap-around, question message, instance messages, 
and response instructions. 

Content file: Authored material, graphics, and audio content. 

Correlation committee: IDT, SME, and PS; meet at various times during the 
production cycle to iron out problems in the lesson and to generate MI 
and answer processing display routines, 

COSD: See OSD« 
\ Courseware: Instructional material. 
Difficulty level: Ascending/descending level of difficulty in a content file 
E: English, 
ED. Editor. 
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EDT: Empirical Design Technician; person responsible for validation of 
instructional materials; a graduate student in the instructional 
psychology program. 

Expository: Description of an example or generality requiring no response 
from the student. 

File reference genulator: Generative strategy for computer generated instance 
and AI files . 

Frame: Forms on which a TV screen is outlined, used to simulate the appearance- 
of the material on the TV. 

Generality: Definition, rule or memory task. 

Grid: Graphic representation of the character spaces existent on a TICCIT 
display. 

Help: A form of mathemagenic information. 
HE.N: Help for non-example. 

Heuristic: Serving to guide, discover, or reveal further information^ 
HE.X: Help for example. 

Hierarchy: Suggested sequence of minor objectives to reach major objectives. 
ICUE: Institute for Computer Uses in Education (BYU) . 

IDT: Instructional Design Technician; advisor for base frame specifications, 
objectives, hierarchies, and other instructional design considerations; 
a graduate student in the Instructional Psychology program. 

Inquisitory: Description of an example that calls for a response from the 
student • 

Instance: A single example or non-example. 
IP: Instructional Psychologist . 

IR and D: Instructional Research and Development. 

IX: Inquisitory example base frame; also known as practice mode. 

LOSD: See OSD. 

M: Pre-calculus course. 

Mathemagenic information: A class of responses which give birth to learning, 
includes heuristic information, algorithms , attribute isolation, and 
mnemonics . 
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Mini-lesson: A shortened version of the regular lesson, giving the students 
- an overview of the generalities and some examples. 

MITRE: The MITRE Corporation. A not-for-profit systems engineering company, 
Subcontractor to NSF on the TICCIT Project. BYU is subcontractor to 
MITRE on the project. 

Mnemonics: Techniques of improving the memory. 

NI4: Non-example message. 

NSF: National Science Foundation 

OSD: Objective Status Display; the display of the objectives on the screen 
for a lesson, course or unit; designated respectively LOSD, COSD, 
and UOSD. Used as an option to preview specific objectives. 

Overlay: A frame with windows that is laid over another frame to help 

illustrate the point and to save having to rewrite the entire screen. 

POHE.N: Post help for non-example. 

POHE.X: Post help for example. 

POSHE.N: Post-super-help for non-example. 

POSHE.X: Pos t~super-help for example. 

PREHE.N: Prehelp for non-example. 

PREHE.X: Prehelp for example. 

PRESHE.N: Pre-super-help for non-example 

PRESHE.X: Pre-super-help for example. 

Packagers: Packaging specialists. 



Production cycle: System through which instructional materials are produced 

from author to packaging and empirical validation prior to its submission 
to MITRE. 

PS: Packaging Specialist. 

QM: Ques.^ion Message^^ 

RE: Requisite English . 

Relevant attributes: Qualitative characteristics which are pertinent to the 
generality . 



RM: Requisite math. 



Senior author: Author who is responsible for the authoring team for that 
course . 

SME: Subject Matter Expert; author. 

So What?: Tells the student why he is learning what he is learning. 
Super-help: Step-by-step presentation of the help. 

Terminal: Place where the student sits: carrel with keyboard, TV screen, 
headphones set included (end of system) . 

TICCIT; Time-shared Interactive Computer-Controlled Information Television 

UOSDt See OSD. 

XM: Example message. 



